The effect of the warping term on the fractional quantum Hall 

states in topological insulators 

Fawei Zheng/ Zhigang Wang/ Zhen-Guo Fu/ and Ping Zhang^'^'Q 

^LCP, Institute of Applied Physics and Computational Mathematics, 
P. 0. Box 8009, Beijmg 100088, China 
^Beijing Computational Science Research Center, Beijing 100084, China 

Abstract 

The warping effect on the fractional quantum Hah (FQH) states in topological insulators is 
studied theoretically. Based on the perturbed wavefunctions, which include contributions from the 
warping term, analytical expressions for Haldane's pseudopotentials are obtained. We show that 
the warping term does not break the symmetry of the pseudopotentials for n=ibl Landau levels 
(LLs). With increasing the warping strength of the Fermi surface, our results indicate that the 
stability of the FQH states for LL n = (LLs n=ibl) becomes stronger (weaker), and the excitation 
gap at 1/ = 1/3 FQH state for LL n=0 also increases while the gaps for LLs n=ibl are unchanged. 
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Topological insulators (TIs) as a new phase of quantum matter, which can not be adi- 
abatically connected to conventional insulators and semiconductors, have been intensively 
studied in recent years iMlOf. TIs are characterized by a full insulating gap in the bulk 
and protected gapless edge or surface states. Near the Fermi level the low-energy dispersion 
of the TI surface states shows a Dirac linear behavior. However, the recent angle-resolved 
p«oe...o„ .,„y exp.™. .how that the Fe... ,n B.Te, g Q. a 

typical TI, is a snowfiake shape rather than a circle one. The origin of this snowflake-like 
ermi surface has been confirmed to arise from an unconventional hexagonal warping term 



It is this warping term that brings about many unique physical phenomena 12h14|. 



which can not be observed in other systems, including the extensively studied graphene and 
the conventional two-dimensional electron gas. 

More recently, there has been emerging attention to the interactions of the Dirac-type 
quasiparticles and their strong correlation effects in TI, especially the TI fractional quan- 
tum Hall (TIFQH) states. Despites no undeniable experimental observation of the TIFQI 
states heretofore, some theoretical studies have been undertaken. For example, DaSilva 15 1 
predicted the stability of the TIFQH states for Landau levels (LLs) with index n=0 and ±1 
in TIs. Apalkov and Chakraborty studied the finite thickness effect on the TIFQH states 
lld l. Also, the present authors investigated the influences of the Zeeman splitting and the 

nn 

tilted strong magnetic field on the stability of the TIFQH states |17l . |18I | with large g fac- 
tor. However, many important and interesting open questions, such as the warping effect, 
the spin excitations, and the subband-LL coupling, have not been discussed in the TIFQH 
regime. 

In this paper we theoretically study the warping effect on the TIFQH states. Here the 
warping term is perturbativly treated. With the help of the numerical calculations, we 



show that the warping term can not break the symmetry of the Ha^ 



for n=±l LLs, which differs from the role of the spin splitting 17|. Moreover, our results 



dane's pseudopotentials 



indicate that with the increase of the warping strength of the Fermi surface, the stability of 
the TIFQH states for LL n = (LLs n=±l) become stronger (weaker), and the excitation 
gap at z/ = 1/3 filling for LL n=0 (gaps for LLs n=±l) also increases (keep unchanged). 
In the presence of a perpendicular magnetic field B = Bz, the effective Dirac Hamiltonian 
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for Bi2Te3(lll) surface is written as 

Ho = Vf {ajly - ayli,) + ^ (n^ - nl) f^., (1) 

where 11 = k+eA/c with the wave vector k = (A;^, ky, 0) and the gauge A = B{—y/2, x/2, 0). 
Here, II-i- = 11^ ±11^, o'x,y,z are Pauh matrices, vj denotes the Fermi velocity, and A describes 



the hexagonal warping strength of the Fermi surface pjj]. Here we have assumed that the 
Zeeman splitting is much weaker than the warping term and therefere can be neglected for 
the first step in order to solely illustrate the role played by the warping term. This is the 
case for Bi2Te3(lll) system. By introducing the ladder operators o.^ = (^-^ ~ S/^Sf j 
and CL = ^ + 2/^(9^^ with z{z)=x ± iy and the magnetic length Ib = ^Jhc/eB, we can 
rewrite the Hamiltonian ([!]) as 




-I (at3 




^0 = ^1 ^ / - ... I- (2) 

Vfa^ 

When A=0 the Hamiltonian ([2]) can be exactly solved, and the eigenstates are given by 

- 1, m) 




for n 7^ 0, 

(3) 

for n = 0, 



where the symbol |n, m) is the non-relativistic two-dimensional electron gas Landau eigen- 
states with non-relativistic quadratic dispersion relation in rath LL with angular momentum 
m. The corresponding LLs are expressed as = sgn {n)vf^j2\n\/lB. When the warping 
term is taken into account (A 7^ 0), however, the single-particle eigenstate can not be ob- 
tained directly. Fortunately, one can take the perturbation method to get eigenstates of 
Hamiltonian (|2]) since the warping term, a/2A/^^, is much smaller than the typical energy 
space between the nearest-neighboring LLs, \/2vf/lB^ i-e., C = A/(f//^) ^ 1. After some 
long but straightforward algebraic operations, and only keeping the first-order terms, we 
have 

Xh+sII^I +2,"^) 
XH-sII^I -3,m) 
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FIG. 1: (Color online) The effective pseudopotentials of the Coulomb interaction between 
two electrons at (a) n=0 and (b) n=l LLs as functions of the relative angular momentum with 
different warping strength (^=0 (circles) and ^=0.2 (stars). 

where the coefficients are 



^(1) __,.,/(H + i)(H 

A|n|+2 ~ '■^^ 



(2) I -^sgn{n)cJ^\^^t2mz2l |^| > 2 
' 1^1 ^ 2 

In the following discussion we will focus our attention to the TIFQH states of \n\ ^ 1, 
because the stab 
pseudopotential 



'IFQH states can only be observed for LLs \n\ ^ 1 [15]. The Haldane's 
19 ] for Coulomb interaction V{v) = ^ between electrons in the nth. LL 



with relative angular momentum m is given by 

yin,m) ^ [^(g)]^ L^(g2/|)e-^ , (6) 

in terms of Laguerre polynomials Lm{x) and the form factor J^{q) = (\I'n|e~*'''''|\E'n) with 
the cyclotron variable rj = r — H. Here, R is the guiding-center position. Explicitly, for LLs 
we have 

Vio,m) ^ Yj^L^{2x)e-'^ [Lo (x) + C'l^2 (x)] ' , (7) 
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FIG. 2: (Color online) The ratio of the first and third relative angular momentum pseudopotentials 
for LLs n=0 (black solid line) and n=ibl (red dashed line) as a function of the warping strength ^. 



and 



2 



^-^ eq 
q 



2x 



(8) 



+ ^(Lo (x) + Li (a;) + 6C% {x)f 

2^2 

with x=—^ being a dimensionless variable. From Eq. (jH]) one can clearly see that the 
Haldane's pseudopotentials for LLs n = 1 and n = —1 are still identical in the presence of 
the warping term. This is different from the Zeeman splitting effect, which can induce an 



asymmetry in the pseudopotentials for n=±l LLs |17l. Il8|. 

Figured] plots the Haldane's pseudopotentials of Coulomb interaction \/("'™) for (a) n=Q 
and (b) n=l LLs as functions of the relative angular momentum m. Comparing with the 
results in the absence of the warping term (black dots in Fig. [1]), one can clearly find that 
in the presence of the warping term, the magnitude of the pseudopotentials increases (red 
stars in Fig. [1]). 

Subsequently, the stability of the TIFQH states should also be modified by the warping 
term. The typical results of V^("'^) /v^C^^s) (^=q g^^id ^—-^i^ g^j.^ g^own in Fig. [2] as a function 
of the warping parameter (. One can see that with increasing \/(0'i)/\/(0'3) increases 
while ^(='=i'i)/\/(='=i'3) decreases. According to the composite fermion theory, the larger the 
value of V"(".i)/V^("=3) ^^e more stable the fractional quantum Hall states. Therefore, the 
remarkable warping term results in the composite fermions at fractional filling for LL n=0 




FIG. 4: (Color online) The increment of the gap width as a function of the warping term (" for 
N=7 electrons at v=\ TIFQH states for LLs n=0 (black solid line) and n=ibl (red dashed line). 

(LLs n=±l) to become more stable (unstable). This result suggests that on the surface of 
a TI material with strong (weak) warping strength, such as Bi2Te3 (Bi2Se3), the TIFQH 
states for LLs n={] (?t,=±1) may be observed much easier under a perpendicular magnetic 
field. We hope this prediction could be detected in future experiment. 

In what follows, by using the exact diagonalization method in the spherical geometry, we 
investigate the system with the fractional filling factor v=l/{2p + 1), where p is an integer. 
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For briefness we only illustrate the z/=l/3 TIFQH state, which is realized at 5* = | (A^ — 1) 
in the spherical geometry with being the electron number. Under this configuration, the 
perpendicular magnetic field is equivalent to a fictitious radial magnetic field produced by 
a magnetic monopole at the center of a sphere of radius R = y/Sls (in unit of fiux quanta), 
and the many-body states could be described by the total angular momentum L and its z 
component L^. 

We show in Fig. [3] the energy spectra of the many-body states at z/=l/3 filling for N=l 
electrons. Comparing the two cases with ((^=0.2) and without (C=0) warping term, one 
can see from Fig. [3] that the gap width at n={] LL between the ground state and the first 
excited state has a visible increment while those at n=±l LL keep unchanged. Furthermore, 
we calculate the corresponding excitation gap width Eg [Q by increasing the warping term 
from C=0 to C=0.3. The variation ^Eg = Eg {() ~ Eg {( = 0) as a function of ( are plotted 
in Fig. m which shows that the TIFQH gap between the ground state and the lowest excited 
state at n=0 LL (solid line) are sensitively dependent on the warping term while AEg at 
n=±l LLs (dashed line) keeps a constant no matter the warping term is included or not. 
This also implies that the warping term is different from the Zeeman splitting and the tilted 
ma gnet ic field, which will induce a change in the gap of the TIFQH states at LLs n=±l 



mm 

In summary, we perturbatively studied the effect of the warping term on the TIFQH 
states. It was found that differing from the role of the Zeeman splitting and the tilted 
magnetic field, the warping term does not break symmetry of the Haldane's pseudopotentials 
for n=±l LLs. Our results showed on one hand that, the stability of the z/ = 1/3 TIFQH 
states for LL n = (LLs n=±l) become stronger (weaker) by increasing the warping 
strength of the Fermi surface. On the other hand, the excitation gap for LL n=0 increases 
with increasing the strength of the warping term, while the gaps for LLs n=±l are yet 
insensitive to this term. 
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